Flying camera with string assembly for localization and interaction

ABSTRACT

According to a first aspect of the present invention there is provided an arrangement comprising, a volitant body comprising at least one actuator; a control unit for controlling said actuator; and a mechanical arrangement for operationally connecting said volitant body to a reference point remote from said volitant body. There is further provided a corresponding method for operating such an arrangement.

FIELD OF THE INVENTION

The present invention relates to aerial imaging, particularly to aflying camera, equipped with rotary wings and a string assembly, and toits localization, stabilization, and interaction therewith.

BACKGROUND

Today the operation of most digital cameras remains restricted toviewpoints that are within an arm length's reach from the operator.

A variety of solutions, both flying and non-flying, have been proposedto solve this problem. However, current flying solutions are typicallydifficult to control, requiring pilot training and experience tosuccessfully obtain an image from a desired view point. They are alsohighly susceptible to wind and prone to loss and collisions with highobjects such as trees, power lines, or houses. Existing non-flyingsolutions, such as extendable poles, rely on ad hoc support elementslike poles, masts, or base stations that lead to complicated, expensive,unwieldy, or otherwise impractical solutions. They are typicallyinexpensive and easy to use, but are cumbersome to transport andseverely restrict the possible viewpoints.

BRIEF SUMMARY

In accordance with the present invention, limitations of previousmethods for aerial imaging have been substantially reduced oreliminated. In particular, the present invention aims to provideimproved systems and methods for position and attitude stabilization forvolitant bodies including flying cameras using a string assembly. Inaddition, the present invention, aims to provide an improved userinteraction for volitant bodies including flying cameras.

In the context of the present invention, such user interaction typicallyenables the translation of user intention to perform an action with, orchange the operation of, a volitant body. This is typically achievedthrough specific physical actions by the user (e.g., pulling a string)and through specific physical actions by the volitant body (e.g.,pulling back on the same string). User interaction in the context of thepresent invention includes, but is not limited to a force applied by auser, or by the volitant body or by both, the activation of switches orbuttons by the user, visual or audible signals from the user or from thevolitant body, or pre-coded sequences and patterns of the above.

According to a first aspect of the present invention there is providedan arrangement comprising, a volitant body comprising at least oneactuator; a control unit for controlling said actuator; and a mechanicalarrangement for operationally connecting said volitant body to areference point remote from said volitant body.

The volitant body preferably comprises a means for flying.

Preferably at least one actuator is configured to actuate said means forflying.

Preferably, the volitant body further comprises a camera.

The operational connection may be of mechanical nature.

Preferably, the mechanical arrangement mechanically connects thevolitant body to the reference point.

The mechanical arrangement may be operationally connected to saidcontrol unit.

The arrangement may comprise an evaluation unit which is operable toprovide data representative of at least one of (a) an attitude; or (b) aposition of said volitant body relative to said reference point, to saidcontrol unit, and wherein said control unit is configured to controlsaid at least one actuator based on said data.

In the present application, attitude means the full definition of therotation of a rigid body relative to an inertial frame, i.e. the body'sorientation in all three dimensions. An example inertial frame is thecommonly used local gravity-ground referenced East-North-Up frame.

Preferably the evaluation unit is comprised in the mechanicalarrangement and is preferably located on the volitant body. Said dataprovided by the evaluation unit is preferably based on mechanical forcesapplied to said mechanical arrangement.

Preferably the evaluation unit is located on the volitant body.

The arrangement may further comprise a sensing unit operable to providedata representative of mechanical forces applied to said mechanicalarrangement to said evaluation unit. Preferably the mechanicalarrangement comprises the sensing unit.

Preferably the sensing unit is located on the volitant body.

The sensing unit may be mechanically connected to said volitant body.

The arrangement may comprise a force sensor, for determining saidmechanical forces, operationally connected to the sensing unit.Preferably the mechanical arrangement comprises the force sensor.

Preferably the force sensor is located on the volitant body.

The arrangement may comprise a sensor for providing data representativeof at least one of an acceleration, attitude, or rotational rate of thevolitant body, which is operationally connected to the sensing unit. Thearrangement may comprise a sensor, for providing data representative ofsaid position of said volitant body relative to said reference point,which is operationally connected to the sensing unit.

The arrangement may comprise a memory unit operationally connected tosaid evaluation unit, in which is stored first data related toproperties of said mechanical arrangement and second data related toproperties of said volitant body, wherein said evaluation unit isconfigured for carrying out an evaluation using at least one of saidfirst or second data, to provide said data representative of at leastone of (a) an attitude or (b) a position of said volitant body relativeto said reference point.

The arrangement may further comprise an active safety means, and/orpassive safety means.

The arrangement may further comprise an active means, and/or passivemeans of user interaction.

Preferably the mechanical arrangement defines a means of userinteraction.

According to a further aspect of the present invention there isprovided, the use of any of the above-mentioned mechanical arrangementsas a communication channel based on mechanical forces applied to themechanical arrangement.

According to a further aspect of the present invention there isprovided, the use of any of the above-mentioned mechanical arrangementsfor aerial imaging.

According to a further aspect of the present invention there isprovided, a method for operating any of the above-mentioned mechanicalarrangements, said method comprising a step of, controlling said atleast one actuator to make said volitant body fly remote from saidreference point.

Preferably the step of controlling said at least one actuator is done,using the mechanical arrangement.

The method may comprise the additional steps of,

-   -   using an evaluation unit to provide data representative of at        least one of (a) an attitude; or (b) a position of said volitant        body relative to said reference point; and    -   providing said data to the control unit, and wherein said        control unit performs controlling of said at least one actuator        based on said results of said data evaluation.

The method may comprise the steps of,

-   -   sensing mechanical forces applied to said mechanical        arrangement; providing the evaluation unit with data which is        representative of mechanical forces applied to said mechanical        arrangement, and wherein the evaluation unit performs the step        of using said data which is representative of mechanical forces        applied to said mechanical arrangement to provide data        representative of at least one of (a) an attitude; or (b) a        position of said volitant body relative to said reference point.

The method may further comprise the steps of,

-   -   memorizing first data related to properties of said mechanical        arrangement, and    -   memorizing second data related to properties of said volitant        body, and performing an evaluation at the evaluation unit, using        at least one of said first or second data, to provide said data        representative of at least one of (a) an attitude; or (b) a        position of said volitant body relative to said reference point.

The method may further comprise the steps of,

-   -   sensing mechanical forces applied to said mechanical        arrangement; and    -   evaluating the sensed mechanical forces to provide evaluation        results;    -   providing the control unit with the evaluation results, and        wherein said step of controlling said at least one actuator is        carried out based on said evaluation results.

Preferably the mechanical forces are applied to the mechanicalarrangement at the reference point.

Preferably, the evaluation results may comprise data representative ofat least one of: magnitude of force, a change in force magnitude,direction of force, and/or a force sequence.

At least one of the steps of evaluating or controlling may comprise thestep of a user interacting with the mechanical arrangement.

Preferably, the step of a user interacting with the mechanicalarrangement comprises the user applying one or more forces to themechanical arrangement or the user reducing the force which is appliedto the mechanical arrangement.

The step of sensing mechanical forces may comprise, sensing at least oneof, direction, magnitude, or time sequence of mechanical forces.

The method may comprise the step of,

-   -   communicating with said reference point by applying one or more        forces to said mechanical arrangement, wherein the said at least        one actuator is controlled by the control unit so that the        volitant body applies said one or more forces to the mechanical        arrangement.

The step of controlling said at least one actuator may include at leastone of executing emergent maneuvers, executing active maneuvers,detecting continuous user input, or detecting parallel user input.

According to a further aspect of the present invention there is provideda method of calculating at least one of attitude or relative position ofa volitant body with respect to a reference point, implemented in anevaluation unit. The method may comprise a first step of, receivingfirst data from an inertial sensor mounted on said volitant body.Optionally the method may comprise a second step, of retrieving seconddata from the memory unit comprising prior data or calculations. Themethod may comprise the step of using the said first and/or second datato (a) predict the instantaneous motion of the vehicle, and (b) tocalculate an approximate position of the volitant body relative to areference point and/or the attitude of the volitant body. Optionally,this prediction and calculation can be improved using prior knowledgeabout the dynamics of the arrangement. Additional sensors, such asstring length sensors, pressure sensors, range sensors or other sensors,may be used to further enhance the measurement, in a computationallyrigorous manner. The method may comprise the step of mathematicallycombining the instantaneous motion prediction and approximatemeasurements to produce at least one of a volitant body attitude and/orvolitant body position estimate with respect to a reference point.

The above evaluation unit provides technical advantages for certainembodiments of the present invention, including allowing for eliminatingdependence on any outside infrastructure such as the visible orradio-magnetic beacons or GPS systems. Furthermore, said evaluation unitin combination with said mechanical arrangement enables furtherperformance-enhancing features such as continuous calibration ofactuator or flight parameters or estimation of external forces such aswind.

According to a further aspect of the present invention there is provideda method for the stabilization or controlled flight of a volitant body,wherein said volitant body comprises a control unit operationallyconnected to said volitant body and at least one actuator configured toactuate the means of flying. The method may comprise, receiving at acontrol unit data from said evaluation unit wherein the data specifiesat least one of the attitude of the volitant body, the instantaneousmotion of the volitant body, or the position of the volitant body withrespect to a reference point. The method may comprise operating saidcontrol unit to access a memory unit to retrieve stored information.Preferably the retrieved information may include actuator calibration,flight parameters, predefined control behaviors or trajectories. Themethod may comprise the step of operating said control unit to calculateappropriate commands for at least one actuator and dispatching commandsto said actuator(s). In addition, said control unit may return saidactuator commands back to the said evaluation unit.

Preferably, the volitant body is the volitant body of the arrangementaccording to any one of the above-mentioned arrangements.

The step of combining the method implemented in said control unit withthe methods implemented by said evaluation, sensing and/or memory units,and with the constraints imposed by said mechanical arrangement providestechnical advantages. These advantages include favorable systemstability qualities, better robustness against effects such as windgusts, or better robustness against rapid movements of the referencepoint. Said combination allows the volitant body to maintain controlledflight even under challenging conditions such as when attached to arapidly moving reference point such as a skier, a boat, or a car.

According to another aspect of the present invention there is provided amethod for interacting with a volitant body, wherein said volitant bodycomprises,

-   -   a control unit operationally connected to said volitant body;    -   and a mechanical arrangement for operationally connecting said        volitant body to a reference point remote from said volitant        body, said method comprising the step of,    -   receiving, at the control unit, data relating to at least one of        direction, magnitude, or time sequence of forces applied to said        mechanical arrangement,    -   wherein the control unit performs the step of controlling the        volitant body so that volitant body performs a predefined        maneuver, wherein the predefined maneuver is associated with the        data it receives. Preferably, the volitant body is the volitant        body of the arrangement according to any one of the        above-mentioned arrangements.

According to yet another aspect of the present invention there isprovided a method for interacting with a volitant body, wherein saidvolitant body comprises,

-   -   a control unit operationally connected to said volitant body;    -   an evaluation unit operationally connected to said volitant        body; and    -   a mechanical arrangement for operationally connecting said        volitant body to a reference point remote from said volitant        body, said method comprising the step of,    -   receiving, at the evaluation unit, data relating to at least one        of direction, of magnitude, or of forces applied to said        mechanical arrangement, wherein    -   the evaluation unit evaluates said data, in combination with        data from said memory unit, to computationally detect specific        sequences or patterns of at least one of direction, of        magnitude, or of forces.    -   The evaluation unit may simultaneously command the control unit        to alter its operation to, for example, provide haptic feedback        to the user during user interaction. The result of the        evaluation by the evaluation unit may be sent to the control        unit to optionally perform at least one of a change of internal        states, the execution of specific actions, or an alteration the        current operating mode of the volitant body. Preferably, the        volitant body is the volitant body of the arrangement according        to any one of the above-mentioned arrangements.

The step of combining said evaluation unit, control unit, and mechanicalarrangement therefore allows for controlled, user-interactive flight ofa volitant body without the need for radio communication or complexconfiguration and programming. The volitant body may be launched andoperated intuitively, with simple tasks such as aerial photographyimplemented through natural user hand gestures applied to the string. Inaddition, the said combination enables a novel channel of communicationfor the volitant body to communicate information back to the user, suchas for example by effecting a specific pattern or sequence of forces onthe string given a certain condition, such as a certain battery level orcompletion of a given task such as a panoramic photographic survey.

Technical advantages of certain embodiments of the present invention mayallow even inexperienced users of all ages to safely capture imageryfrom a wide range of viewpoints without the need for ad-hoc supportelements. For example, the present invention may allow minimizing oreliminating risks inherent in current aerial imaging arising fromcollisions, mechanical or electrical failures, electronic malfunctions,operator errors, or adverse environmental conditions, such as wind orturbulence.

Other technical advantages of certain embodiments of the presentinvention may allow easier operation in a wide variety of operatingconditions and environments. This may allow even novice users, youngusers, or automated/semi-automated/user-controlled base stations toperform tasks currently performed by experienced human pilots with bothmanned and unmanned flying vehicles. The need for human pilots severelylimits the cost-effectiveness, possible operating conditions, and flightendurance of flying vehicles in many applications. For example, evenexperienced human pilots cannot guarantee safe and efficient control inmany real-world operating conditions including wind and turbulence.

Yet other technical advantages of certain embodiments of the presentinvention may allow it to be tailored to the specific needs of a varietyof applications in a variety of contexts. Example applications includehobbyist platforms for communities such as DIY Drones; researchplatforms for groups actively researching flying platforms or using themas part of their curriculum; military use with requirements such assurvivability, power autonomy, detectability, or operation in extremeconditions (weather, lighting conditions, contamination); toys such assmall flying vehicles; stage performances including choreographies setto music and light or theater performances which require interactionwith theater actors; recreational use similar to that of kites;industrial or public service applications (e.g., surveillance andmonitoring of industrial sites, photogrammetry, surveying); professionalaerial photography or cinematography; or inspection and monitoring ofcivil infrastructure, which may require dangerous or repetitive tasks,in particular, certain technical advantages allow the present inventionto be equipped with a wide range of sensors. For example, infraredsensors allow embodiments for detection of patches of dry ground inorchards or for crop monitoring.

Other technical advantages of the present invention will be readilyapparent to one skilled in the art from those following figures,descriptions, and claims. Moreover, while specific advantages have beenenumerated above, various embodiments may include all, some, or none ofthe enumerated advantages.

BRIEF DESCRIPTION OF THE FIGURES

The present invention is described, by way of example only, withreference to the following figures, which illustrate exemplaryembodiments of the invention:

FIG. 1 shows a user controlling a first embodiment of a flying camerausing a string;

FIG. 2 shows a block diagram of an example control method;

FIG. 3 shows a block diagram of the flight module and its parts;

FIG. 4 shows a flowchart for explaining the process of using the flyingcamera;

FIG. 5 shows a block diagram for explaining the user interaction withthe flying camera by using a string;

FIG. 6 shows another embodiment of a flying camera with a safety shroudaround a counter-rotating propeller configuration and a stereo camerasetup.

DETAILED DESCRIPTION OF FIGURES

To facilitate an understanding of the present invention, it is describedhereinafter with particular reference to a series of specific exemplaryembodiments. It will be appreciated, however, that the principles whichunderlie the invention are not limited to these particular embodiments.Rather, these principles can be combined with many systems to localize,stabilize, interact with, and control volitant bodies using a mechanicalarrangement.

In addition, various features of the invention are occasionallydescribed in the context of their implementation for flying cameras andfor the sample application of aerial imaging. These features are equallyapplicable to other types of volitant bodies, other sensors, and otherapplications. Consequently, specific aspects of the implementation thatare described hereinafter should not be viewed as any limitation on theapplicability of the invention to these volitant bodies, these sensors,or to any specific application.

Overview and Typical Mode of Use

FIG. 1 shows an example of a typical use scenario for the inventionsdescribed herein. A user 108 holds the user end of a mechanicalarrangement (here a simple string assembly consisting of a single string106). The string 106 is connected to a volitant body (here a flyingcamera 102) that lifts and stabilizes itself using a typical helicopterconfiguration: a propulsion/stabilization system 104 comprising a main,swash plate-equipped rotary wing for lift and lateral stabilization, anda small auxiliary tail propeller for yaw stabilization.

The string 106 enables the flying camera 102 to perform preciselocalization and stabilization relative to a reference point 126 (herenear the user 108). In addition, the string 106 enables the user 108 tocommunicate with the flying camera 102 (and vice versa). Such userinteraction may, for example, comprise pulling the string into a desireddirection. This interaction can be achieved by, for example, using adirectional force sensor 124 mounted on the flying camera 102 andattached to the string 106, and the flight module 128 (comprising, e.g.,sensing units, evaluation units, memory units, or control units) whichare attached to the body of the flying camera 102.

To facilitate explanation of the modes of interaction with the flyingcamera 102, the application in FIG. 1 is taken as aerial imaging: thatis, the user is attempting to accomplish the end goal of collectingimagery from a desired viewpoint, typically from an elevated position,as enabled by the flying camera 102 equipped with a camera 110.

Localization and Stabilization

The present invention substantially reduces or eliminates the problemsof recovering the attitude and the position of the flying camera 102,relative to the global coordinate system 114 and to the reference point126, respectively. For clarity, the following explanation discusses thetwo dimensional (2D) case. However, the invention generalizes to, and isuseful in, three spatial dimensions (3D).

The principal forces 120 acting on the center of mass 122 of the flyingcamera 102 are the force of gravity F_(g), the force of propulsion F_(p)generated by the rotary wings, and the force of the string F_(s) pulledby the user 108. Angle α 116 denotes the angle between the user 108 andthe flying camera 102. Angle β 118 denotes the angle of the direction ofthe force of gravity F_(g) and the “up” direction z_(b) of the flyingcamera 102.

On a typical volitant body, such as a flying vehicle, not attached tothe ground, inertial sensors can not, to first order, provide absolutemeasurements of the angles α 116 and β 118. Second-order effects such asaerodynamic drag may enable such measurements in some conditions but donot provide angle information for hover flight. The present inventionallows to overcome this limitation.

This may be achieved as follows. The flying camera 102 is equipped withsensors providing data representative of at least one of anacceleration, attitude, or rotational rate of the flying camera 102(e.g., magnetometers, accelerometers, gyroscopes) and connected to thefixed coordinate system 114 using a string 106. Assuming a taut string106, accurate sensor measurements, and a known geometry of the flyingcamera 102, one formulates the mathematical model of the flying device'ssetup. In particular, one exploits the knowledge of the basic forces 120acting on the flying camera's 102 center of mass 120, the mechanicalconstraints imposed by a taut string 106, and the resulting algebraicrelationships between α 116, β 118, F_(p), and F_(g) as shown in FIG. 1.

The resulting accelerometer measurements a_(x) and a_(z), aligned withthe body reference frame (x_(b), z_(b)), respectively, area _(x) =−F _(s) sin(α−β)a _(z) =F _(p) −F _(s) cos(α−β)

Since the string is taut, the forces in the direction of the string mustbe equal:F _(s) =F _(p) cos(α−β)−g cos a+l{dot over (α)} ²where l is the string length,

The above mathematical model, describing the measured specific forcesacting on the body as a function of α 116, β 118, F_(p), F_(s), andF_(g), is then inverted either in closed-form or numerically, e.g. bysampling the model at regular intervals. The resulting indirectobservation of the F_(g) vector through the inertial sensors provides anabsolute measurement of the gravity vector, and, therefore, angle β 118.In other words, the above method allows an evaluation unit to determinethe direction of gravity based on the physical constraints imposed bythe string and its inertial sensors, which in turn allows a control unitto stabilize the flying camera's 102 attitude and remain airborne andcontrollable.

In one approach, we may assume that the centripetal term l {dot over(α)}² above is negligible. Given a nominal F_(p) and accelerometermeasurements a_(x) and a_(z), we may recover the tension force F_(s) andthe two angles:

$F_{s} = \sqrt{a_{x}^{2} + {( {a_{z} - F_{p}} )^{\bigwedge}2}}$$\alpha = {{acos}( \frac{{{F_{p}( {F_{p} - a_{z}} )}/F_{s}} - F_{s}}{g} )}$β = α − asin(−a_(x)/F_(s))

Note that there is a sign ambiguity in this calculation; the evaluationunit may resolve this ambiguity by referencing the previously estimatedvalues of α and β, and also by using other sensors, e.g. angle rate orstring angle sensors, to provide a priori estimates at a given time.

When combined with additional information of the string length orvehicle altitude, such as from a barometric altimeter, this techniqueenables the evaluation unit to recover the relative position between areference point 126 and a flying camera 102 by exploiting the simpletrigonometric relationships of lengths and angles (see FIG. 1 for the 2Dcase). This may be used by the control unit to e.g., control the flyingcamera 102, aim the camera 110, or a combination thereof.

A refinement of this invention allows more precise localization if aforce or direction sensor 124 is installed attaching the string 106 tothe flying camera 102. In particular, such a sensor can provide data tobe used by an evaluation unit to provide both a more robust estimate ofa 116 and 118 as well as improve user interaction with the flyingplatform. More particularly, if the height of the flying camera 102 orthe length of the string 106 is known (e.g., stored in a memory unit),or unknown but held fixed, these angle estimates can be exploited toproduce an attitude and/or position estimate using the evaluation unit.Specifically, the position and/or attitude may be partial (e.g.,position on a circle or position along a line in a specific direction;direction in a plane or tilt along a single axis) or full 3D positionand/or attitude. They may allow a control unit to actively stabilize theflying camera's 102 attitude and/or position relative to the referencepoint 126. This is achieved by exploiting the trigonometricrelationships of lengths and angles in 3D (see FIG. 1 for the 2D case).

In addition, the pose estimation techniques disclosed here may becombined with other measurements such as from a satellite-based GlobalPositioning System, beacon-based position and attitude measurementsystems, or others. For example, data from an additional yaw-attitudesensor, such as a magnetometer, mounted on the flying camera 102 may beused to allow an evaluation unit to produce a full 3D position andattitude estimate. This is achieved by exploiting the trigonometricrelationships of lengths and angles, including the angle of thevehicle's attitude with respect to the user or another reference framesuch as GPS coordinates in 3D. This may be especially useful inapplications that require the flying camera to face in a given directionindependent of the user's movement.

The method described above is not limited to specific flightconfigurations, such as the helicopter configuration shown in FIG. 1.The analysis leading to the recovery of the angles α 116 and β 118 canbe shown to have good robustness qualities under proper operatingconditions, meaning the described invention remains useful under windconditions or when the reference point 126 is moving, such as being heldby a moving person or attached to a moving vehicle or attached to amoving base station. Furthermore, although sensing is explained withreference to inertial sensors, the method described above is equallyapplicable when using other sensors providing data representative ofattitude or position. Moreover, the method is not limited to use of asingle string and many other mechanical arrangements are possible.

FIG. 2 shows a block diagram of an example control method that may beused to stabilize the flying camera 102. During operation of the controlmethod, a numerical method (the “state estimator” 204, typicallyimplemented in an evaluation unit) is used to form an estimate of thestate of the flying camera 102 from the measurements of a sensor forproviding data representative of at least one of an acceleration,attitude, or rotational rate of the flying camera 102 and, optionally, aforce sensor acting on the string attachment point 124 of the flyingcamera. In addition, a memory unit may be used. Depending on thespecific requirements and use case, methods that may be used to form theestimate known in the prior art include one or multiple LuenbergerObservers, Kalman Filters, Extended Kalman Filters, Unscented KalmanFilters, and Particle Filters, and combinations thereof. Proper onboardsensor fusion (typically implemented in an evaluation unit) can alsoprovide operational fault robustness against invalid model assumptions,such as a momentarily relaxed string, providing valid short time-frameestimates and allowing for emergency procedures to be activated ifnecessary.

Based on the state estimate 204, a feedback control method 206, 208(typically implemented in a control unit) provides control inputs to theat least one actuator 210, which, depending on the embodiment of theinvention, may consist of rotary wings, swash plates, control surfaces,or other means to apply forces and/or torques to the flying camera 102.The feedback control is designed to control the position 206 andattitude 208, as described by the body coordinate system 112 and mayconsist of parallel or cascaded feedback control loops. Depending on thespecific requirements and use case, methods that may be used to computethe flight actuator control signal include linear or nonlinear statefeedback, model predictive control, and fuzzy control.

As an example embodiment, consider a vehicle operating in the vertical2D plane, accepting as commands a nominal collective thrust F_(p), and adesired angular rate {dot over (β)}*. One possible control law tomaintain a desired string angle ᾰ is a cascaded controller, where adesired string angle acceleration {umlaut over (α)}* is first computed,to be then translated into a desired vehicle angle β*, which in turnyields {dot over (β)}*:

${\overset{¨}{\alpha}}^{*} = {\frac{1}{\tau_{s}^{2}( {\overset{\bigvee}{\alpha} - a} )} - {\frac{2\zeta_{s}}{\tau_{s}}\overset{.}{\alpha}}}$

Where τ_(s) and ζ_(s) are tuning factors, corresponding to the desiredtime constant of the closed-loop string angle system and the desireddamping ratio. Then the desired vehicle angle and angle rate may becomputed as follows:β*=a−a sin((g sin α−lä*)/F _(p)){dot over (β)}=1/τ_(V)(β*−β)where τ_(V) is also a tuning parameter.

The state estimate 204, in particular the recovered value of F_(s), mayalso be used as a communication channel or to detect the userinteracting with the flying camera. A user command detection 212 detectsmatches of the state information (e.g., vehicle is airborne, vehicle isfilming) with predefined or computationally learnt interaction patterns(e.g., three quick tugs on the string, two strong tugs followed by along pull sideways). Based on the detected patterns, the feedbackcontrol methods (typically implemented in a control unit) 206, 208 areprovided with data representative of the command that the user 108provided. The command is also forwarded to a camera control system 214,therefore allowing the user 108 to independently control the behaviourof the flying camera 102 and the camera 110. The user interaction systemimplemented in the user command detection system 212 is further detailedin the following section.

FIG. 3 shows a block diagram of the flight module 128 and its parts,including a control unit 302, evaluation unit 304, sensing unit 306 andmemory unit 308. The flight module receives sensor information as inputand provides data to the actuators.

Depending on the specific requirements and use case, multiple evaluationunits, sensing units, memory units, and control units may be used.Similarly, multiple steps (e.g., both, steps relating to datarepresentative of position and/or attitude as well as steps relating todata representative of user interactions) may be performed in a singleunit.

Sensing units are used to process sensor information. For example, theymay process information received from sensors, such as accelerometers,gyroscopes, magnetometers, barometers, thermometers, hygrometers,bumpers, chemical sensors, electromagnetic sensors, or microphones (noneshown). For example, a sensing unit may process information to extractdata on forces caused by the mechanical connection of a volitant body,such as a flying camera, to the reference point. Such forces may forexample be caused by the vehicle being restrained by the string, by auser tugging on the string or by disturbances, such as those caused bywind or motor malfunction. A sensing unit may also be used to detectpatterns in data representative of mechanical forces applied to amechanical arrangement, such as a string in order to realize acommunication channel using the mechanical arrangement. In addition,sensing units may process information from one or multiple cameras 110,608 aboard the flying camera 102.

Evaluation units are used to evaluate data. For example, they mayevaluate data representative of both relative or absolute position,particularly that of GPS sensors, visual odometry/SLAM, retro-reflectivepositioning systems, laser range finders, WiFi positioning systems,barometric altimeters and variometers, or ultra-sound sensors (noneshown). For example, they may use data representative of mechanicalforces provided by the sensing unit to infer data representative ofposition and attitude of a volitant body, such as a flying camera,relative to a reference point.

Memory units are used to store data. For example, they may be used tostore data on past sensor readings, operational states or user commands,as well as properties of the flying camera 102 and string 106.

Control units are used to control actuators. For example, they may allowactive (and passive) self-stabilization by generating control outputsfor the flight actuators (e.g. the air propulsion system 104) as well asother actuators (e.g., camera 110 zoom/pan/tilt motors). These controloutputs may be generated in dependence of data representative ofposition and attitude provided by the evaluation unit

User Interaction

Direct measurement via an optional force sensor 124 or estimation of theforce F_(s) exerted by the string 106 may enable an evaluation unit 304to detect user interaction with the flying camera 102 via physicalforces applied to the string 106. Optionally, to allow for communicationfrom the flying camera 102 to the user, a control unit 302 may beconfigured to allow user interaction by controlling at least oneactuator of the flying camera 102.

FIG. 4 shows a flowchart of an exemplary user interaction process withthe flying camera 102 for the sample application of aerial imaging.After turning on the power switch, the flying camera 102 performs a selfcheck 402. The check is designed to ensure correct, safe operation ofall units and actuators of the flying camera 102 before it becomesairborne. For example, it may comprise checks that all components haveinitialized correctly, that the flying camera responds to stringcommands, or that battery levels are sufficient for flight. In addition,the self check 402 may comprise calibration steps, such as positioningthe device on a level surface to calibrate the internal inertial sensorsor adjusting the white balance for the attached camera 110. The selfcheck 402 may further comprise other checks, such as hanging the devicefrom the string 106 to determine if it has a suitable weight and weightbalance with respect to its center of mass 122 and string attachmentpoint 124, or if the position of the camera 110 or the battery pack (notshown) needs to be adjusted.

After a successful self check 402 the flying camera is ready for userrelease 404. User release 404 may be performed in a variety of ways,depending on the size, weight, passive and active safety features, orits sensory capabilities. For example, a user may first hold the flyingcamera 102 above their head, then command it to spin up its propellers104 by shaking it twice (typically detected by an evaluation unit 304),and then release a handle 602 with an integrated safety switch 604. Inother cases, the flying camera 102 may be started or operated from abase station placed on the ground (not shown), or held in the user'shand, or it may be started by simply tossing it higher than two metersinto the air and allowing it to auto-stabilize.

The flying camera 102 then enters a Hover Mode 406, where a control unit302 stabilizes its position by sending appropriate commands to theactuators in dependence of data received from an evaluation unit 304.Stabilization may be relative to a reference point, the string, the air,the ground, or a combination thereof.

In the exemplary flowchart shown in FIG. 4, the flying camera is thenready to perform maneuvers based on user commands. Such maneuvers may betriggered by detecting representative data using a sensing unit 306 andprocessing said data, optionally together with data from a memory unit308, in an evaluation unit 304.

Maneuvers are sequences of desired vehicle positions, attitudes, andactions and are typically stored in a memory unit 308. Maneuvers aretypically executed by sending appropriate commands to the actuators independence of data received from an evaluation unit 304, Simple examplesinclude “Take photo”, “Move towards tug”, or “Land”. However, maneuverscan also comprise more complex routines, such as “Take panoramic photo”,which involves controlling the vehicle to a sequence of positions andtaking a sequence of images. Unlike the example shown in FIG. 4,maneuvers may also comprise entire operations performed without userinput. For example, a user may select settings on a user interface thatcause the flying camera to perform a maneuver that involves autonomoustake off, flying to a set point relative to the reference point 126,taking a series of photographs, and finally returning to the launchposition for autonomous landing.

Once the user issues a command to land 410, typically detected by anevaluation unit 304, the control unit 302 commands actuators such thatthe flying camera 102 performs a landing maneuver 412 and exits theprogram.

FIG. 5 shows an exemplary embodiment of a simple state machine for userinteraction with the flying camera 102 (typically implemented in anevaluation unit 304). When the flying camera 102 is hovering, the user108 on the ground may move the reference point 126 laterally, causingthe flying camera 102 to perform a lateral shift maneuver 502. The usermay also release or retract the string to change the flying camera'saltitude 504, Furthermore, the user may use a signal, such as a shorttug 508 on the string (typically detected by an evaluation unit 304based on data from at least one sensing unit 306), to trigger a certainaction such as taking a photograph or video. Finally, the user may usesignals, such as sideways tugs, to tell the flying camera 102 toreorient 506. For example, a tug signal may be detected by an evaluationunit 304 based on data provided by a sensing unit 306 and, optionally,data provided by a memory unit 308, and cause the control unit 302 tosend commands to actuators such that a reorientation occurs. Similarly,the vehicle may use signals to communicate with the user, e.g. insteadof the reorientation in the example above, a control unit 302 may sendcommands to actuators such that a communication signal (such as a tugsignal) is sent to the user.

User interaction input typically detected by the evaluation unit 304 cancomprise any combination of the direction, magnitude, and time sequenceof forces applied to the string 106 (e.g., at its end or at thereference point 126), It can be measured via a force sensor at thestring attachment point 124 or via other sensors (e.g., accelerometers,rate gyros), It can be identified by user command detection methods 212.This setup allows for a rich and intuitive user interface, significantlyextending existing interfaces, such as those known from mouse clicks(which do not account for direction and magnitude), or mouse gestures(which do not account for magnitude).

Furthermore the flying camera 102 may provide a communication channelbased on mechanical forces (e.g., haptic feedback such as mechanicalstimulation used in tactile feedback) for communication between a basestation at the reference point or for user interaction via a string.This may be used for bi-directional communication, for example, tocommunicate to the reference point whether a command has been recognizedand whether a command has been entered correctly. To realize such acommunication channel, the onboard controller of the flying camera can,for example, command a series of small jerks, using the propulsionsystem, to indicate a particular haptic message back to the user via thetaut string. As another example, the flying camera could execute acontinuous series of upward jerks, in a clearly perceivable time-forcepattern, to indicate a warning or error condition such as low battery orfull onboard memory. For example, the string may be used as acommunication channel for user commands to be sent from the user to theflying camera, or user feedback to be sent from the flying camera to theuser, based on mechanical forces applied to the string.

The flying camera can also provide visual feedback via light displays orby intentionally performing clearly interpretable motions as a visualresponse to user commands, providing the user a means of learning thenecessary commands and the contexts they can be used in. Both haptic andvisual feedback are key components for intuitive user interfaces.Depending on the size and proximity of the flying camera, such clearlyinterpretable maneuvers may be further improved by adding suitableauditory cues, further improving the reliability of user interaction.Haptic, visual, and auditory feedback can be generated using the flyingcamera's main actuators (e.g., its motors) or additional actuators(e.g., vibrators, linear actuators, lights, speakers) or a combinationof both.

Maneuvers can be active or passive. Active maneuvers are triggered bythe User Command Detect 212 and governed by the flying camera's controlmethod (FIG. 2). For example, the flying camera 102 may perform a singlefull controlled rotation on the spot following a user command. Passivemaneuvers are purely the result of user interaction with the physicaldynamics, determined by the flying camera, the string assembly, and itsfeedback control system. They emerge from the stabilization behavior ofthe flying vehicle and take advantage of the robustness of the standardstate estimation and control behavior to change the state of the flyingcamera. For example, a user may release additional string and therebyallow the flying camera to fly higher.

Maneuvers may treat user input as binary or continuous. For example, adetected change in the magnitude of force on the string may only triggera maneuver if it passes a threshold. Continuous user input may alsoserve as a parameter. For example, the magnitude of force on the stringmay be used as an indication for the desired amount of side movement orthe number of photos to be taken.

Maneuvers can be executed sequentially or in parallel. For example, theflying camera may not accept further commands before completing asequence of photos suitable for assembly of a panoramic image. Or, auser may issue a lateral movement command 502 and simultaneously retractthe string, constraining the flying camera to a spiraling motion towardsthe user.

It should be noted that while the above descriptions list only a smallnumber of user interaction cases with exemplary embodiments by way ofexample, the possible user input commands via the string, flying cameraresponses via the string, and possible maneuvers of the flying cameragive rise to a large number of possible input-output mappings and userinteraction scenarios. Given the benefits of the present disclosure, oneof ordinary skill in the art can devise suitable communication schemesinvolving user input or various communication channels (including basedon mechanical forces, etc.) for a wide range of applications, between awide range of entities (including for users, base stations, etc.), andfor a wide range of volitant bodies other than flying cameras.

Further Exemplary Embodiments

FIG. 6 shows another exemplary embodiment of a flying camera 102. Theembodiment of FIG. 6 is equipped with various active and passive safetyfeatures, including a handle 602 for launch/retrieval convenience; asafety switch 604 mounted on the handle to enable quick, safe launches:and a light but rigid cage 606 surrounding the propulsion system 104.

Various other safety features may be employed, depending on therequirements and use case. In particular, the flying camera 102 may beequipped with additional passive (e.g., non-conductive string assembly,pre-determined breaking points, crash-safe propellers, redundantcomponents) and active (e.g., autonomous obstacle avoidance,auto-stabilization, autonomous take-off/landing, audible alarm, visualerror indicator, deadman switch, emergency kill switch, backup flightcontrol routines (automated/autonomous “co-pilots”)) safety features(none shown) to reduce risks to the operator, to their environment, andto the flying camera 102.

The embodiment of FIG. 6 is equipped with a propulsion system comprisingtwo coaxial, counter-rotating propellers. One or both of the propellersmay be equipped with a mechanism, such as a swash plate, or anothermeans of producing controlled torque, such as a control momentgyroscope, to produce the required forces and torques to control theflying camera 102.

Moreover, various other propulsion systems may be employed, depending onthe requirements and use case. In particular, the flying camera 102 maybe equipped with flybars; passively stable rotor assemblies, such asthose described in U.S. Pat. No. 7,494,320; or quadrocopter, hexacopter,or other multi-rotor configurations (none shown) to increase efficiencyand redundancy.

The embodiment of FIG. 6 shows a stereo camera setup 608 fordepth-enabled imaging or 3D telepresence applications. Generally, thecameras mounted on the flying camera 102 may be any type of sensorreceptive to electromagnetic radiation, including those with activesensors (e.g., structured light and projection systems).

In the embodiment of FIG. 6, the flight module 128 is positioned on theflying camera 102. However, depending on the requirements and use case,some or all of its components including control units, sensing units,evaluation units, and memory units, may also be positioned elsewhereincluding at the reference point 126.

For some embodiments additional functionalities, such as wirelesscommunication, onboard image processing, or battery-heating circuitry(none shown) may be added as required. Furthermore, other properties ofa string 106 may be exploited, such as using it as a fiber-opticcommunication medium, using it as a bending sensor, using it to liftother types of sensor, or as a sensor for measuring properties, such asstrain or twist.

Although the description above contains many specificities, these shouldnot be construed as limiting the scope of the embodiments but as merelyproviding illustrations of some of several embodiments. For example,lift can be achieved using a large variety of means includingpropellers, airfoils, ducts, and vanes; said means can be assembled inmany configurations including fixed-wing, rotary-wing, and ornithopters;strings can have various elasticity or comprise sensing elements. Thusthe scope of the embodiments should be determined by the appended claimsand their legal equivalents, rather than by the examples given.

REFERENCE NUMBERS

-   102 Flying camera-   104 Air propulsion system-   106 String-   108 User-   110 Camera-   112 Body coordinate system-   114 Global coordinate system-   116 α (Alpha) (angle user to flying camera)-   118 β (Beta) (angle flying camera to gravity)-   120 Forces acting on flying camera-   122 Center of mass of the flying camera-   124 Force sensor/string attachment point-   126 Reference point-   128 Flight module-   202 Inertial sensors-   204 State estimation-   206 Position control-   208 Attitude control-   210 Flight actuators-   212 User command detect-   214 Camera control-   216 Memory-   218 Camera actuators-   302 Control unit-   304 Evaluation unit-   306 Sensing unit-   308 Memory unit-   402 Perform self-check-   404 User release-   406 Hover-   408 User command-   410 Land-   412 Landing maneuver-   500 Hover-   502 Move laterally-   504 Move up/down-   506 Orient-   508 Take photo-   602 Handle-   604 Safety switch-   606 Protective cage-   608 Stereo camera

What is claimed is:
 1. An actuator-controlled volitant body comprisingone or more physical processors configured to: determine a tension alonga communication channel established between the actuator-controlledvolitant body and a reference point based on an acceleration of theactuator-controlled volitant body.
 2. The actuator-controlled volitantbody of claim 1, wherein the communication channel is established via atether connecting the actuator-controlled volitant body to the referencepoint.
 3. The actuator-controlled volitant body of claim 2, wherein theone or more physical processors are further configured to: obtain firstinformation representative of one or both of an attitude or a positionof the actuator-controlled volitant body relative to the referencepoint, wherein the attitude of the actuator-controlled volitant body isdetermined based on the tension and the acceleration; and control anactuator of the actuator-controlled volitant body based on the firstinformation.
 4. The actuator-controlled volitant body of claim 3,wherein the one or more physical processors are further configured to:obtain second information representative of forces applied to thetether; and evaluate the second information to determine the firstinformation.
 5. The actuator-controlled volitant body of claim 4,further comprising: a sensor, the sensor being configured to generatemeasurements for determining the forces.
 6. The actuator-controlledvolitant body of claim 3, further comprising: a sensor, the sensor beingconfigured to generate measurements representative of the position ofthe actuator-controlled volitant body relative to the reference point.7. The actuator-controlled volitant body of claim 3, further comprising:a memory coupled to the one or more physical processors, the memorybeing configured to store second information related to properties ofthe tether and third information related to properties of theactuator-controlled volitant body; and wherein the one or more physicalprocessors are further configured to: perform an evaluation using one orboth of the second information or the third information; and generatethe first information representative of one or both of the attitude orthe position of the actuator-controlled volitant body relative to thereference point.
 8. The actuator-controlled volitant body of claim 2,wherein the one or more physical processors are further configured to:receive information relating to one or more of a direction, a magnitude,or a time sequence of forces applied to the tether; and control theactuator-controlled volitant body so that the actuator-controlledvolitant body performs a predefined maneuver, wherein the predefinedmaneuver is associated with the information.
 9. The actuator-controlledvolitant body of claim 1, further comprising: a sensor, the sensor beingconfigured to generate measurements representative of one or more of theacceleration, an attitude, or a rotational rate of theactuator-controlled volitant body.
 10. The actuator-controlled volitantbody of claim 1, wherein the actuator-controlled volitant body furthercomprises a camera, and wherein the one or more physical processors arefurther configured to capture images using the camera.
 11. A method tooperate a volitant body comprising an actuator and one or more physicalprocessors, and wherein a communication channel is established betweenthe volitant body and a reference point, the method comprising:controlling the actuator of the volitant body to fly remote from thereference point; and determining a tension along the communicationchannel using an acceleration of the volitant body.
 12. The method ofclaim 11, wherein the communication channel is established via a tetherconnecting the volitant body to the reference point.
 13. The method ofclaim 12, further comprising: obtaining first information representativeof one or both of an attitude or a position of the volitant bodyrelative to the reference point, wherein the attitude of the volitantbody is determined based on the tension and the acceleration; andcontrolling the actuator of the volitant body based on the firstinformation.
 14. The method of claim 13, further comprising: obtainingsecond information representative of forces applied to the tether; andevaluating the second information to determine the first information.15. The method of claim 14, further comprising generating measurementsfor determining the forces.
 16. The method of claim 13, furthercomprising generating measurements representative of the position of thevolitant body relative to the reference point.
 17. The method of claim13, further comprising: storing second information related to propertiesof the tether and third information related to properties of thevolitant body; performing an evaluation using one or both of the secondinformation or the third information; and generating the firstinformation representative of one or both of the attitude or theposition of the volitant body relative to the reference point.
 18. Themethod of claim 12, further comprising: receiving information relatingto one or more of a direction, a magnitude, or a time sequence of forcesapplied to the tether; and controlling the actuator so that the volitantbody performs a predefined maneuver, wherein the predefined maneuver isassociated with the information.
 19. The method of claim 11, furthercomprising generating measurements representative of one or more of theacceleration, an attitude, or a rotational rate of the volitant body.20. The method of claim 11, wherein the volitant body further comprisesa camera, and the method further comprises capturing aerial images usingthe camera.